18.05 Lecture 24 
April 8, 2005 



Bayes Estimator. 

Prior Distribution f(8) — > compute posterior f(9\Xi, ...,X n ) 
Bayes's Estimator = expectation of the posterior. 
K(X — a) 2 — ► minimize a — > a = EX 

Example: B(p),f(p) = Beta(a,/3) -> / (p|xi, x„) = Beta(a + Y^, x u + n ~ H x i) 

6(x!, ...,x n ) = a ~^^ Xt 
a + p + n 

Example: Poisson Distribution 

\ x 



II(A),/(.x|A) = 
Joint p.f.: 



ft \ r£ . \ \ ^ . 

f( Xu ..., Xn \X)=]J—e^ = — T e-^ 



Xi< Uxil 

i=i 



If /(A) is the prior distribution, posterior: 



T^fe-" A /(A) 

JVM \ llXi'. J \ J 



g(x 1 ...x n ) = f^£e-'*f(\)d\ 
Note that g does not depend on A: 

f(\\ Xl ,...,x n )~ \Z*'e- nX f(\) 
Need to choose the appropriate prior distribution, Gamma distribution works for Poisson. 



Take /(A) - p.d.f. of T(a,P), 



f(X\x u x n ) ~ AS x i+a -i e -{n+p)\ _^ r(a + Xu + n) 
Bayes Estimator: 

\( Xl ,...,x n )=EX= a+ ^ Xi 
n + p 

Once again, balances both prior intuition and data, by law of large numbers: 

\{x\,...,x n ) = i + pj n co x^ E(Ai) -> A 

The estimator approaches what you're looking for, with large n. 

Exponential E(a), f(x\a) = ae~ ax , x > 
f( Xll ... lXn \a) = IL? =1 ae- ax > = a n e-^ x ^ a 
If /(a) - prior, the posterior: 
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f(a\ Xl ,...,x n )~a n e-^^ a f(a) 



Once again, a Gamma distribution is implied. 
Choose /(a) — T(u, v) 



11 

f( a ) = a u ~ 1 e~ va 



New posterior: 

f(a\ Xl , x n ) ~ a "+«- 1 e -(S^+-)« -^T(u + n,v + J2 &i) 



Bayes Estimator: 



, . u + n u/n+1 > 1 1 

a(a;i,...,a; n ) = — = — -= r n ^ oo - ^ — = a 



Normal Distribution: 

NM,f{x\^cr) = -^=e-^-^ 
(TV27T 

f{ Xl ,...,x n \n,a) = —L—e-^^fr-rf 
((7v27rj™ 

It is difficult to find simple prior when both /i, a are unknown. 
Say that er is given, and fi is the only parameter: 

Prior: /(/x) = — ^ e -^- a ) 2 = JV(o,6) 
6v2-7r 



Posterior: 

/(M|Xi,..,X n )~e-3^S(xi-M) 2 -5iT(M-) a 

Simplify the exponent: 

= i £(* 2 - 2 ^ + + i^ 2 - ^ + fl2 ) = ^ + i) - 2 ^i? + £) + 

= [i 2 A 2»B + ... = A(fi 2 2„§ + (| ) 2 ) - ^ + - = - | f + ... 

/^|Ai,...,A„j e e iv^,^_j iV( ^ + ^ , ^ + 

Normal Bayes Estimator: 

. cr 2 a + nb 2 x a 2 a/n + b 2 x > _ . 

H (X 1} ...,X n ) = — 5- — ro- = — T — -Tj- n ^ oo a; ^ E li = ^ 



End of Lecture 24 
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